Workpiece pick-up apparatus

ABSTRACT

A workpiece pick-up apparatus including: a hand for gripping a workpiece; a robot for bringing the hand into a desired gripping position or posture; a sensor for performing three-dimensional measurement of the workpiece to obtain workpiece measurement data; a storage medium for accumulating at least hand profile data; an information processing unit for calculating the gripping position or posture based on data from the sensor and data from the storage medium; and a control unit for controlling the robot based on the gripping position or posture calculated by the information processing unit. The information processing unit includes an optimum gripping candidate creating section for directly deriving the gripping position or posture based on the workpiece measurement data and the hand profile data.

TECHNICAL FIELD

The present invention relates to a workpiece pick-up apparatus forpicking up a workpiece from among bulked workpieces.

BACKGROUND ART

The workpiece pick-up apparatus is an apparatus for picking upworkpieces one by one through use of a robotic hand from an objectcontaining a plurality of bulked workpieces. As a conventional workpiecepick-up apparatus, there is an apparatus disclosed in, for example,Patent Literature 1. This apparatus is configured to assume arepresentative profile of a workpiece, prestore data on partial profileunits, which are obtained by segmenting the workpiece, and prioritiescorresponding to the respective partial profiles, perform imageprocessing on a plurality of bulkedworkpieces so as to calculate aplurality of partial profiles as candidates for a gripping position, anddetermine, from among the calculated candidates, a workpiece as anobject to be picked up and a gripping portion thereof in considerationof the above-mentioned priorities.

In the above-mentioned method of picking up a workpiece as disclosed inPatent Literature 1, however, the following problems may arise.

As a first problem, under the condition that the number of calculatedcandidates for a gripping position is small, when the hand is to gripthe workpiece determined from among the candidates, there is such a highrisk that the hand cannot reach the gripping position due tointerference between the hand and the workpieces other than the objectto be gripped, or to disturbance of the other workpieces. In order toavoid this problem, it is necessary to calculate a large number ofcandidates for a gripping position in advance, but in this case, therein turn arises a problem in that the labor, calculation time, and dataamount are increased considerably.

As a second problem, the operations of obtaining the partial profilesthrough segmentation and assigning the priorities to the respectivepartial profiles need to be defined newly for the profiles of handsdifferent from each other. For this reason as well, there arises aproblem in that the labor and calculation time are increased.

CITATION LIST Patent Literature

[PTL 1] JP 2010-89238 A

SUMMARY OF INVENTION Technical Problems

The present invention has been made in view of the above-mentionedcircumstances, and it is therefore an object thereof to provide aworkpiece pick-up apparatus capable of picking up a workpiece whileachieving a small amount of data to be held in advance and a shortcalculation time.

Solution to Problems

In order to achieve the above-mentioned object, according to the presentinvention, there is provided a workpiece pick-up apparatus, including: ahand for gripping a workpiece; a robot for bringing the hand into agripping position or posture that is desired; a sensor for performingthree-dimensional measurement of the workpiece to obtain workpiecemeasurement data; a storage medium for accumulating at least handprofile data; an information processing unit for calculating thegripping position or posture based on data from the sensor and data fromthe storage medium; and a control unit for controlling the robot basedon the gripping position or posture calculated by the informationprocessing unit, in which the information processing unit includes anoptimum gripping candidate creating section for directly deriving thegripping position or posture based on the workpiece measurement data andthe hand profile data.

Advantageous Effects of Invention

According to the workpiece pick-up apparatus of the present invention,through use of the same algorithm even when a workpiece profile haschanged, the workpiece can be picked up while achieving a small amountof data to be held in advance and a short calculation time.

BRIEF DESCRIPTION OF DRAWINGS

[FIG. 1] An apparatus configuration diagram illustrating a workpiecepick-up apparatus according to a first embodiment of the presentinvention.

[FIG. 2] A diagram exemplifying hand profile data and hand profilemodels based on the hand profile data.

[FIG. 3] An apparatus configuration diagram illustrating a workpiecepick-up apparatus according to a second embodiment of the presentinvention.

[FIG. 4] An apparatus configuration diagram illustrating a workpiecepick-up apparatus according to a third embodiment of the presentinvention.

[FIG. 5] An apparatus configuration diagram illustrating a workpiecepick-up apparatus according to a fourth embodiment of the presentinvention.

[FIG. 6] Adiagram illustrating apart of a workpiece pick-up apparatusaccording to a fifth embodiment of the present invention.

[FIG. 7] Diagrams illustrating restriction on operations of a robot anda hand in picking up a workpiece from among bulked workpieces.

[FIG. 8] A diagram illustrating a relationship between a line-of-sightdirection of a sensor and a direction of an entrance operation.

[FIG. 9] A diagram illustrating an example of a distance image.

[FIG. 10] A diagram illustrating a camera image and a distance imageobtained through measurement of a bulked state of workpieces with athree-dimensional sensor.

[FIG. 11] A diagram exemplifying a two-dimensional hand model to be usedin the fifth embodiment.

[FIG. 12] A conceptual diagram illustrating processing of creating anoptimum gripping candidate through use of the distance image and thetwo-dimensional hand model.

[FIG. 13] A flow chart illustrating the processing of FIG. 12.

[FIG. 14] A diagram illustrating a clearer resultant image of a grippingposition or posture in FIG. 12.

[FIG. 15] An explanatory diagram illustrating a state in which anopening/closing direction of the hand at a gripping position isperpendicular to directions of profile edges of a candidate segment sothat a gripping state is more stable.

[FIG. 16] An apparatus configuration diagram illustrating the workpiecepick-up apparatus according to the fifth embodiment.

[FIG. 17] A graph showing results of evaluating a success rate by amethod according to the fifth embodiment when picking up workpieces forwhich a gripping test has not been conducted due to difficulty inrecognizing the respective workpieces by a conventional method based ona workpiece profile.

DESCRIPTION OF EMBODIMENTS

In the following, a workpiece pick-up apparatus according to embodimentsof the present invention is described with reference to the accompanyingdrawings. Note that, in the figures, the same reference symbolsrepresent the same or corresponding parts.

First Embodiment.

FIG. 1 is an apparatus configuration diagram illustrating a workpiecepick-up apparatus according to a first embodiment of the presentinvention. The workpiece pick-up apparatus includes at least a storagemeduim 1, a sensor 2, an information processing unit 3, a control unit4, a robot 5, and a hand 6.

The storage meduim 1 accumulates at least hand profile data. The sensor2 is configured to obtain three-dimensional measurement data of bulkedworkpieces. The information processing unit 3 outputs a grippingposition or posture for a workpiece based on the data from the storagemedium 1 and the sensor 2. The control unit 4 is configured to controlan operation of the robot 5 and an operation of the hand 6 based on thedata on the gripping position or posture obtained by the informationprocessing unit 3. Further, the robot 5 is configured to bring the hand6 into an arbitrary position or posture based on a command from thecontrol unit 4, and the hand 6 is configured to grip the workpiece.

The sensor 2 may be installed at a tip of the robot 5, or installed atanother appropriate fixing position than the robot 5. Further, thesensor 2 may be, for example, a twin- or multi-lens stereo camera, anactive stereo camera including a light projecting unit, such as a laserand a projector, and a camera unit, a device using the time-of-flightmethod, a single-lens device using a robot operation based on thefactorization method, the structure-from-motion process, and thestructure-and-motion process, a motion stereo camera, and a device usingthe volume intersection method, as long as the sensor 2 can obtainthree-dimensional data in an arbitrary region.

The hand 6 may be a hand of an external gripping type, asuction/attraction type, or a type in which the hand is inserted into ahole and opened (hereinafter referred to as “internal gripping type”).

One of the features of the present invention resides in that thegripping position or posture is dynamically determined based on the handprofile and the measurement data instead of determining the grippingposition or posture in advance based on the workpiece profile.Specifically, the information processing unit 3 includes an optimumgripping candidate creating section 30 for dynamically determining agripping position in the field represented by the workpiece measurementdata from the sensor 2. Further, the optimum gripping candidate creatingsection 30 includes a grippable feature extracting section 301, a handmatching section 302, and a gripping posture candidate selecting section303. The grippable feature extracting section 301 extracts grippablefeatures from the workpiece measurement data of the sensor 2. The handmatching section 302 matches the hand profile data accumulated in thestorage meduim 1 with the features extracted by the grippable featureextracting section 301 to dynamically create a plurality of grippingposition/posture candidates. The gripping posture candidate selectingsection 303 selects, from among the gripping position/posture candidatescreated by the hand matching section 302, a gripping position orposturewhich enables the easiest gripping. In the following, therespective sections are described in more detail.

The grippable feature extracting section 301 extracts features grippableby the hand 6 in use irrespective of the entire workpiece profile. Forexample, in a case of the hand 6 of the external gripping type, thefeatures correspond to protrusions or edges which are easy to pinch. Ina case of the hand of the suction/attraction type, the featurescorrespond to a surface in a given range or wider. Further, in a case ofthe hand of the internal gripping type, the features correspond to acircular hole formed in the surface of the workpiece.

Those features can be extracted by fitting an edge, a surface, a circle,and the like to the three-dimensional measurement data. As a method fordetermining those features at high speed, for example, there are amethod of detecting edges in a distance image showing a distancerepresented by brightness through use of the Canny operator, the Sobeloperator, or the like, a method of extracting a circle through the Houghtransform, a method of extracting a surface by labeling a regionsurrounded by edges, and a method of detecting a textured portion asprotrusions through use of an edge gradient direction of the brightnessand intensity thereof. Those features are determined through basic imageprocessing, and hence the processing is completed at considerably highspeed even when a large number of features are extracted.

The hand matching section 302 matches the features extracted by thegrippable feature extracting section 301 with the hand profile data(model) accumulated in the storage meduim 1. The matching is implementedby providing models as illustrated in, for example, FIG. 2 in accordancewith the types of the hand. For example, in the case of the externalgripping type, the matching may be defined by an opening width of thehand arm immediately before gripping approach, a depth of entrance, anda longitudinal width and a lateral width of the hand. Further, in thecase of the suction/attraction type, the matching may be defined by aradius of the suction or attraction, and in the case of the internalgripping type, the matching may be defined by a radius of the insertionhole, a radius of a workpiece surface on the periphery of the hole, andthe depth of entrance. The hand profile data as described above hassimple parameters, and hence the necessary data amount is small. Whendetermining the parameters, the parameters may be calculated based onCAD data of the hand 6, or the hand 6 may be measured directly.

In the matching with the hand profile model as described above, forexample, in the case of the suction/attraction type, the candidate isset to a portion having a high degree of matching between the model andthe feature point group, but a matching score of the candidate isdecreased when the degree of matching with the model is low due to asmall surface of the feature or a hole formed in the surface of thefeature. Ina case of a 3D model, this processing is implemented bymatching between the point groups using the iterative closest point(ICP) method or the like, and in a case of a 2D model, this processingis implemented by template matching or matching using convolutionprocessing with the model regarded as a filter. Further, in the case ofthe external gripping type and the internal gripping type, interferencewith the surrounding environment may further be taken into considerationat the same time. Specifically, for example, in the case of the externalgripping type, when the measurement data is included in a region definedby the depth of entrance and the longitudinal width and the lateralwidth of the hand, this processing is implemented by decreasing thematching score. The hand matching section 302 calculates grippingpositions or postures each having the highest matching score for theplurality of features extracted by the grippable feature extractingsection 301, and sets the plurality of gripping positions or postures asthe gripping position/posture candidates. Alternatively, the handmatching section 302 calculates a plurality of gripping positions orpostures each having a matching score higher than a predeterminedthreshold value for a single feature extracted by the grippable featureextracting section 301, and sets the plurality of gripping positions orpostures as the gripping position/posture candidates. Through theprocessing described above, the gripping positions or postures can bedefined dynamically even when the workpiece profile is not defined.

Further, in the above-mentioned matching, the grippable featureextracting section 301 calculates a principal normal on the distanceimage, and hence the score can be calculated through two-dimensionalimage matching with three degrees of freedom, specifically, two degreesof freedom for translation and one degree of freedom for rotation. Thus,it is possible to calculate the gripping positions or postures whichenable easy gripping at high speed without interference between thefeatures.

The gripping posture candidate selecting section 303 selects, from amongthe gripping position/posture candidates created by the hand matchingsection 302, a gripping position/posture candidate which enables theeasiest gripping. The gripping posture candidate selecting section 303may select a gripping position/posture candidate having the highestscore assigned by the hand matching section 302, and may further assignadditional priorities. For example, the gripping posture candidateselecting section 303 may use an average height and gravity centerpositions of the features extracted by the grippable feature extractingsection 301 to select a workpiece located at the highest position in thescene of the bulked workpieces.

When a single optimum gripping position or posture is created by theinformation processing unit 3 as described above, the control unit 4controls the operation of the robot 5 and the operation of the hand 6based on the data on the gripping position candidate, and the hand 6grips and picks up the targeted workpiece from among the plurality ofbulked workpieces.

According to the workpiece pick-up apparatus of this embodiment that isconfigured as described above, when at least the hand profile data andthe hand profile models based on the hand profile data are held aspreparatory information, the gripping position or posture, which is lessliable to cause the interference between the peripheral workpieces andthe hand and enables easy gripping, can be calculated dynamicallyirrespective of the workpiece profile and the state of the scene of thebulked workpieces. Thus, through use of the same algorithm even when theworkpiece profile has changed, the workpiece can be picked up whileachieving a small amount of data to be held in advance and a shortcalculation time. Further, there is no need to perform processing ofredefining the priorities of the parts of the workpiece every time thehand profile has changed, and hence it is possible to resolve theproblem in that the labor and calculation time are increased along withthe change of the priorities.

Second Embodiment.

In the above-mentioned first embodiment, the hand profile data isdirectly matched with the workpiece measurement data, that is, only thehandprofile data is used for calculating the optimum gripping positionor posture. In contrast, in a second embodiment of the presentinvention, the gripping positions or postures are narrowed down, andthen workpiece profile data is further used for estimating a state ofentanglement between the workpieces and estimating whether or not thepick-up operation is successful. Such estimation cannot be performedwhen only the measurement data is used.

In the second embodiment, the storage medium 1 also accumulates theworkpiece profile data in addition to the hand profile data, and asillustrated in FIG. 3, the information processing unit 3 furtherincludes a workpiece state determining section 31 for evaluating, basedon the workpiece profile data, the gripping positions or posturescalculated by the optimum gripping candidate creating section 30.

As illustrated in FIG. 3, the workpiece state determining section 31includes a workpiece matching section 311, a workpiece entanglementstate estimating section 312, a workpiece pick-up operation estimatingsection 313, and a gripping posture candidate selecting section 314. Theworkpiece entanglement state estimating section 312 is configured toimplement an operation of determining interference between workpiecemodels, which are created through matching between the workpiecemeasurement data and the workpiece profile data, so as to determineentanglement between the bulked workpieces, which is not observed in theworkpiece measurement data, and giving preference to gripping positionsor postures corresponding to objects with less entanglement. Theworkpiece pick-up operation estimating section 313 is configured toimplement an operation of calculating gravity center positions of therespective workpieces at the time of gripping based on the hand profiledata, the workpiece profile data, and the calculated gripping positionsor postures, and giving preference, based on the gravity centerposition, to gripping positions or postures with a lower risk of fallingof the workpiece after the gripping or inclination of the workpieceafter the gripping.

In the following, operations of the respective sections are described inmore detail. First, the optimum gripping candidate creating section 30of the second embodiment is basically the same as the optimum grippingcandidate creating section 30 of the first embodiment, but is differentin that the optimum gripping candidate creating section 30 of the secondembodiment selects a plurality of gripping positions or postures insteadof determining only one gripping position or posture which can beevaluated as being optimum. In this case, scores for assigningpriorities may be created for the plurality of selected grippingpositions or postures.

The gripping positions or postures newly calculated by the optimumgripping candidate creating section 30 and the extracted features thatare used for the calculation are set as initial candidates, and theworkpiece matching section 311 matches the initial candidates with theworkpiece profile data. The fact that the initial candidates havealready been determined corresponds to the completion of initial searchperformed for the matching with the workpiece models. In this case, whenthe features are constituted by three-dimensional point groups or edgepoint groups, accurate matching is performed by the ICP method or thelike. In the case of a two-dimensional image, there may be employedtemplate matching with three degrees of freedom, specifically, twodegrees of freedom for translation and one degree of freedom forrotation, matching using hashes, silhouette matching, or matching basedon a geometric relationship between the features. Further, portions ofthe workpiece which are likely to be extracted as the features by theoptimum gripping candidate creating section 30 may be determined inadvance to narrow the search range for matching, which leads to higherspeed matching.

The workpiece entanglement state estimating section 312 obtainscorresponding workpiece models by matching the workpiece measurementdata with the workpiece profile data for a workpiece corresponding to agiven gripping position or posture and a workpiece group on theperiphery of the workpiece. The workpiece entanglement state estimatingsection 312 then analyzes the states of those workpiece models todetermine the interference between the workpieces. As a result, whenparts of the respective workpiece models are located at the sameposition so that the workpiece models interfere with each other, theworkpiece entanglement state estimating section 312 performs, forexample, processing of excluding the gripping positions or posturescorresponding to those workpiece models from the candidates, orprocessing of decreasing the priorities of candidate selection in thegripping posture candidate selecting section 314. In this case, evenwhen the workpiece models do not interfere with each other, theworkpiece entanglement state estimating section 312 may simulate thepick-up operation for the workpiece models, and when the workpiecemodels abut against other peripheral workpiece models as a result of theoperation, the workpiece entanglement state estimating section 312 maysimilarly exclude the corresponding gripping positions or postures fromthe candidates or decrease the priorities thereof.

The workpiece pick-up operation estimating section 313 simulates thegripping by using a given gripping position or posture and acorresponding workpiece model, and further using a corresponding handprofile model at the gripping position or posture. When the hand gripsthe workpiece at the gripping position or posture, the workpiece pick-upoperation estimating section 313 calculates a specific gravity centerposition of the workpiece, but excludes or decreases the priority of thegripping position or posture which exhibits a high risk of falling atthe time of the pick-up operation. In order to perform this processing,the workpiece pick-up operation estimating section 313 may determine therisk of falling by, for example, setting an evaluation index to anEuclidean distance between the gripping position of the hand prepared asillustrated in FIG. 2 and the calculated gravity center position of theworkpiece.

The gripping posture candidate selecting section 314 selects an optimumgripping position or posture based on the priorities evaluated by theworkpiece entanglement state estimating section 312 and the workpiecepick-up operation estimating section 313. In this case, the scoresassigned to the candidates created by the optimum gripping candidatecreating section 30 may be used. The respective evaluation indices maybe represented by linear combination such as addition or non-linearcombination such as multiplication. Further, there may be employed, forexample, a selection method of referring mainly to the scores of theoptimum gripping candidate creating section 30 and discarding candidateshaving the priorities of the workpiece entanglement state estimatingsection 312 and the workpiece pick-up operation estimating section 313which exhibit values equal to or smaller than a given value.

According to the workpiece pick-up apparatus of the second embodiment,similarly to the first embodiment, a candidate which is less liable tocause the interference with the hand and enables highly accurategripping can be selected while reducing the data amount, the labor forregistering the models, and the calculation time. In addition, in thisembodiment, a gripping position or posture which is less liable to causethe entanglement between the workpieces and enables the workpiece to bepicked up with a lower risk of failure of falling during the pick-upoperation is preferentially selected, and hence a pick-up operation witha higher success rate can be implemented as well.

Third Embodiment.

Referring to FIG. 4, a workpiece pick-up apparatus according to a thirdembodiment of the present invention is described. The third embodimentis configured so that, in the above-mentioned second embodiment, thestorage meduim 1 further accumulates data on a gripping position orposture in the subsequent work or operation, and as illustrated in FIG.4, the information processing unit 3 further includes a subsequentwork/operation estimating section for estimating a gripping positioncandidate suited to the subsequent work.

A subsequent work/operation estimating section 32 estimates the grippingposition candidate based on the gripping position or posture in thesubsequent work or operation, which is accumulated in the storage meduim1. In this estimation, when the work subsequent to the work of pickingup the workpiece is, for example, assembly of the workpiece to a productunder the assembling process, the gripping position or posture for theworkpiece that is required during the assembly is limited to a grippingposition or posture suited to the assembling operation. For example, theworkpiece which is gripped in a reversed posture cannot directly beassembled in many cases, and further, it is difficult to largely changethe posture of the robot which is taken during the gripping. Aminorgripping error is allowable, but when the gripping is performed with amajor error or in a posture of the workpiece different from theassembling posture, the workpiece gripping posture needs to be changed,resulting in an increase in working time and labor. Therefore, in thisembodiment, when it is determined that the workpiece gripping postureneeds to be changed for the subsequent work from the workpiece grippingposture during the pick-up operation, the subsequent work/operationestimating section 32 decreases the priority of the grippingposition/posture as a candidate or discards the grippingposition/posture as a candidate. Note that, the subsequent work hereinrefers to, for example, transportation, palletization, and packaging aswell as the assembly.

As a method of determining whether or not the workpiece gripping postureneeds to be changed from the gripping position or posture during thepick-up operation, the subsequent work/operation estimating section 32simulates a gripping state of the workpiece through use of the workpieceprofile model, the hand profile model, and the gripping position orposture. As a specific example, the subsequent work/operation estimatingsection 32 matches the workpiece profile model in a state of beinggripped by the hand profile model with the workpiece profile model in astate of being gripped by the hand profile model at the workpieceposition which is obtained by simulating the subsequent work such as theassembly. When the robot can compute a transformation matrix between thepositions or postures of the respective hand profiles under a state inwhich the positions or postures of the respective workpiece profilemodels are matched with each other, the subsequent work/operationestimating section 32 determines that the operation in the subsequentwork can be performed without changing the workpiece gripping posture,and when the robot cannot compute the transformation matrix, thesubsequent work/operation estimating section 32 determines that it isnecessary to additionally perform a work of changing the workpiecegripping posture by, for example, gripping the workpiece again in adifferent posture or transferring the workpiece to another robotic hand.

According to the workpiece pick-up apparatus of this embodiment that isconfigured as described above, similarly to the above-mentionedembodiments, the workpiece can be picked up while achieving a smallamount of data to be held in advance and a short calculation time. Inaddition, the tact time can be reduced by eliminating the work ofchanging the workpiece gripping posture.

Fourth Embodiment.

The third embodiment illustrated in FIG. 4 employs the processing ofscreening, by the workpiece state determining section 31, the pluralityof gripping position/posture candidates which are calculated by theoptimum gripping candidate creating section 30, and then furtherscreening the resultant gripping position/posture candidates by thesubsequent work/operation estimating section 32. In contrast, a fourthembodiment of the present invention employs, as illustrated in FIG. 5,processing of evaluating, by the workpiece state determining section 31and the subsequent work/operation estimating section 32 in a parallelmanner, the plurality of gripping position/posture candidates which arecalculated by the optimum gripping candidate creating section 30, andfinally, comprehensively determining the evaluations by a grippingposture candidate selecting section 33.

In this case, the evaluation may be performed in such a manner that thescores of the candidates calculated by the optimum gripping candidatecreating section 30 and the evaluation values of the workpiece statedetermining section 31 and the subsequent work/operation estimatingsection 32 are represented by linear combination such as addition ornon-linear combination such as multiplication. Further, there mayadditionally be employed, for example, such processing as to interpose aselection method of referring mainly to the scores of the optimumgripping candidate creating section 30 and discarding candidates havingthe evaluation values of the workpiece state determining section 31 andthe subsequent work/operation estimating section 32 which exhibit valuesequal to or smaller than a given value.

According to this configuration, similarly to the third embodiment, thetact time can be reduced by implementing an accurate pick-up work andeliminating the work of changing the workpiece gripping posture. Inaddition, flexible design can be performed for the usage of theevaluation values in accordance with, for example, a part of an actualproduction systemwhich is liable to cause trouble.

Fifth Embodiment.

Description is given of an embodiment for implementing a versatileoperation of picking up a workpiece from among bulked workpieces, whichachieves high speed processing and facilitates adjustment irrespectiveof the workpiece profile. FIG. 6 illustrates a part of a workpiecepick-up apparatus according to a fifth embodiment of the presentinvention. The hand 6 of an external gripping type and the sensor 2 forthree-dimensional measurement are installed at the tip of the robot 5.The robot 5 of FIG. 6 is an articulated robot with six degrees offreedom, but may instead be an articulated robot with seven degrees offreedom. Alternatively, the robot 5 may be a dual arm robot, a verticalscalar robot, or a parallel-link robot. Further, the sensor 2 isinstalled at the tip end of the robot 5 so as to be operable together,but alternatively, the sensor 2 may be fixed and installed separatelyfrom the robot 5, or installed on a different movable stage.

Operations of the robot and the hand in picking up a workpiece fromamong bulked workpieces are restricted as illustrated in FIG. 7. It isassumed that, at a tip end position of the hand 6, the robot 5 has acoordinate system with X-, Y-, and Z-directions as illustrated in FIG.7. In this case, as illustrated in FIG. 7( a), the position or postureof the hand 6 is adjusted above a feed box 7 so that one of the bulkedobjects in the feed box 7 can be gripped through an operation with twodegrees of freedom for translation along the X-axis and the Y-axis andone degree of freedom for rotation about the Z-axis. Then, asillustrated in FIG. 7( b), the hand 6 enters the feed box 7 through amotion with one degree of freedom for translation in the Z-direction togrip the workpiece. Further, as illustrated in FIG. 7( c), the hand 6 israised through an operation with one degree of freedom for translationin the Z-direction while gripping the workpiece. Those operations areimplemented by operations with a total of four degrees of freedom,specifically, three degrees of freedom for translation and one degree offreedom for rotation.

There are many advantages from the above-mentioned operations. Forexample, in a case where the motion with six degrees of freedom fallsout of the operating range of the robot 5, when the robot 5 iscontrolled to perform the motion with six degrees of freedom inaccordance with the posture of the workpiece, the robot 5 stops due toan error. However, when the pick-up operation can be implemented by theabove-mentioned operations with a total of four degrees of freedom, sucha situation can be avoided with high possibility. In addition, it ispossible to avoid a risk in that the robot 5 or the hand 6 collidesagainst the feed box 7, the sensor 2, or the like due to the complexmotion with six degrees of freedom. It is easy to design the robotoperating range, the safety range, and the interference avoidance rangeby the motion with four degrees of freedom. Further, when the operationis performed with a total of four degrees of freedom, a vertical scalarrobot which is cost efficient and operates at high speed may be employedas the robot 5.

When the entrance operation of approaching the workpiece is restrictedas the operation with one degree of freedom for translation, asillustrated in FIG. 8, a line-of-sight direction of the sensor 2 isaligned with the direction of the entrance operation. The line-of-sightdirection of the sensor herein refers to an optical axis direction of acamera lens. In this case, the gripping position or posture can becalculated at high speed through only the image processing based on thedistance image taken by the sensor 2 and the two-dimensional model ofthe hand profile illustrated in FIG. 2 (hereinafter referred to as“two-dimensional hand model”).

FIG. 9 illustrates an example of the distance image. In a case of animage taken by a camera, each pixel holds a light reflection amount onthe surface of the object as a brightness value. On the other hand, inthe case of the distance image, each pixel holds a height of acorresponding portion of the object. The distance image of FIG. 9 showsthat a portion closer to the sensor 2 that has taken the image isbrighter, and a portion spaced apart from the sensor 2 is darker. In thedistance image, three-dimensional information containing a height can behandled as an image, and hence there is an advantage in that thecomputation amount is smaller as compared to the case where theinformation is handled as three-dimensional position data. In addition,in the case of the distance image, many image processing methods thathave conventionally been used in the production field are applicable.The distance image may be obtained by a three-dimensional measurementmethod capable of obtaining height information, such as a CodedStructured Light (spatial coded method) and a stereographic method, anda three-dimensional sensor for implementing the three-dimensionalmeasurement method. FIG. 10 illustrates a camera image and a distanceimage obtained through measurement of a bulked state of given workpieceswith the three-dimensional sensor.

Description is given of a method of creating an optimum grippingcandidate to be used for picking up a workpiece in the above-mentionedconfiguration in which the operation of the robot 5 is restricted as theoperation with four degrees of freedom, the line-of-sight direction ofthe sensor 2 is aligned with the direction of the entrance operation,and the distance image of bulked workpieces is used. Note that, thefollowing description is directed exclusively to the case of picking upand gripping the workpiece irrespective of the gripping posture for theworkpiece. CAD data, point group data, distance data, two-dimensionalimage data, and partial information (area of the workpiece at a specificportion, length of the edges, feature of the texture, geometricrelationship between characteristic portions of the workpiece, such as ahole) of the workpiece are not prepared in advance. This is for thepurpose of reducing the labor required for the preadjustment inaccordance with the workpiece profile, and for avoiding the increase indata amount along with the increase in number of workpieces to behandled.

A two-dimensional hand model as illustrated in FIG. 11 is used forcalculating the gripping position for the workpiece based on thedistance image. This two-dimensional hand model is obtained by furthersimplifying the 2D hand profile model of the external gripping typeillustrated in FIG. 2, and the distal end portions of the hand, whichabut against the workpiece at the instant when the hand 6 performs theentrance and gripping operations, are represented by circularinterference regions. There are two interference regions correspondingto the tip ends of the hand for gripping the workpiece, and hence thetwo-dimensional hand model can be defined by only a total of twoparameters, specifically, the radius of each circle and the openingwidth of the hand, which determines the positional relationship betweenthe two circles. When each of the tip ends of the hand is a rectangularsolid, the two-dimensional hand model may be represented by using threeparameters, specifically, the length and width of the rectangular solidinstead of the radius of the circle. Alternatively, the two-dimensionalhand model may be defined by parameters indicating a polygon or anellipse. However, the two-dimensional hand model does not need to bedefined as strictly as in the case described above. The hand functionscorrectly even when approximate circles are used in the model foravoiding the interference as long as the radius of each approximatecircle is set sufficiently large so as to cover the entire interferenceregion.

FIG. 12 conceptually illustrates processing of creating an optimumgripping candidate through use of the distance image and thetwo-dimensional hand model. FIG. 13 illustrates a processing flowthereof. In this processing, segments are first extracted from thedistance image. The segment herein refers to a flat or curved surfaceportion of the workpiece surrounded by profile edges. The segmentationrefers to the extraction of segments. The extraction is implemented byedge detection in the distance image using the Canny operator or theSobel operator. The edge in the distance image refers to the profileedge of the object itself. The region segmented by the detected edgescorresponds to the flat or curved surface portion of the workpiece. Bythis method, the segments may be extracted also for thin screws,springs, and the like, which are hard to perform model matching with theworkpiece profile.

Subsequently, the large number of extracted segments are narrowed downto several candidates. This narrowing contributes to the reduction incalculation time, but the processing itself may be implemented even whenall the segments are selected as candidates, and hence the presentinvention may be carried out without narrowing the segments. As a methodof selecting candidates, for example, a plurality of segments may beselected in a descending order of priority from a segment located at thehighest position, or the candidates may be selected based on the areasof the segments instead of the height information. From among thosecandidates, there is output a gripping position or posture having thehighest score (likelihood of gripping) resulting from matching asdescribed below.

The segments selected as the candidates are each matched with thetwo-dimensional hand model. In this manner, search is performed for aposition which enables stable gripping without collision. Thisprocessing is implemented by searching for a gripping position orposture in a region which 1) ensures a large area of the segment presentwithin the opening width of the hand, and 2) prevents the interferenceand collision between the interference regions at the tip ends of thehand and the periphery of the segment. The region 1) can be calculatedthrough convolution processing of the candidate segment and the portionof the opening width of the hand, and the region 2) can be calculatedthrough convolution processing of the candidate segment including itsperiphery, which may cause collision, and the interference regions atthe tip ends of the hand. When the region 2) is subtracted from theregion 1), a “region which may enable gripping and has no risk ofcollision” can be obtained. The resultant region is smoothed and thenvisualized as a likelihood of gripping illustrated in FIG. 12. That is,the gripping position or posture to be calculated is a position orposture of the hand model at the time when the maximum likelihood isrecorded. This position or posture is calculated for the only onesegment selected as the candidate or the plurality of segments selectedas the candidates. In the case of the only one segment, there isselected a position or posture of the hand model which records themaximum likelihood in the segment. In the case of the plurality ofsegments, there is selected a position or posture of the hand modelwhich records the highest value of the maximum likelihood in the segmentamong the maximum likelihoods in the respective segments. The resultantimage of the gripping position or posture illustrated in FIG. 12 showsthat the calculation result thus output indicates appropriateness ofgripping which is performed substantially perpendicular to the edge atthe gravity center position of a partially projecting portion of theworkpiece. The value “92” is obtained by normalizing the likelihood ofgripping with respect to the area, and indicates the “easiness ofgripping of the workpiece” on a scale of “100”. The dark gray circle inthe middle represents the gripping position, and the two white circlesconnected through the straight line represent the interference regionsat the tip ends of the hand.

FIG. 14 illustrates a clearer result obtained by calculating thegripping positions or postures for several candidate segments in thesame manner. Referring to FIG. 14, a portion having no interference onthe periphery scores high, and a portion having the risk of collision atthe time of entrance scores low. In addition, it is found that a portionorthogonal to the edge at the gripping position scores high.

Further, in order to stabilize the gripping state, it is preferred thatthe opening/closing direction of the hand at the gripping position beperpendicular to the direction of the profile edge of the candidatesegment. FIG. 15 illustrates an example thereof. Edges of the candidatesegment are detected, and edge directions in the respective localregions are calculated. When the two-dimensional hand model oriented ina given opening/closing direction of the hand is present at a givengripping position which overlaps with the candidate segment, theorthogonality is evaluated higher as the edge direction of the profileedge of the candidate segment, which intersects with the two-dimensionalhand model, and the opening/closing direction of the hand are closer tothe perpendicular state, to thereby search for a more stable grippingposture. Through multiplication of the above-mentioned score of thelikelihood of gripping by the evaluation result for the orthogonality oraddition of the evaluation result thereto with a weight, both theevaluations can be reflected.

FIG. 16 is an apparatus configuration diagram illustrating the workpiecepick-up apparatus according to the fifth embodiment of the presentinvention. The apparatus includes the storage meduim 1 for accumulatingonly the two-dimensional hand model, the sensor 2 for generating thedistance image of bulked workpieces, the information processing unit 3which includes the optimum gripping candidate creating section 30 forcalculating the gripping position or posture based on thetwo-dimensional hand model from the storage medium 1 and the distanceimage from the sensor 2, the control unit 4 for controlling theoperations based on the information on the gripping position or posture,the robot 5 for bringing the hand into an arbitrary position or posturebased on a command from the control unit, and the hand 6 for gripping aworkpiece.

The above-mentioned flow of FIG. 12 is a flow of processing to beperformed by the optimum gripping candidate creating section 30. Thisprocessing is implemented by a PC including a 2G Core 2 Duo and a 2Gmemory within a calculation time of about 0.1 sec to 0.2 sec, and it istherefore found that the processing is performed at considerably highspeed as the processing which handles three-dimensional information.FIG. 17 shows results of evaluating a success rate by a method accordingto the fifth embodiment of the present invention when picking upworkpieces for which a gripping test has not been conducted due todifficulty in recognizing the respective workpieces by the conventionalmethod based on the workpiece profile. The “single object pick-upoperation” refers to a case of picking up only one workpiece, and the“multiple object pick-up operation” refers to a case of picking up aplurality of workpieces at the same time. For the multiple objectpick-up operation, in the subsequent process, there may be taken, forexample, an action of discarding some workpieces based on themeasurement result from the camera or determination on the weight, orseparating the workpieces under a state in which the workpieces areplaced on a flat surface. Therefore, assuming that the failurecorresponds only to a case where the workpieces are not picked up, anaverage success rate of all the workpieces is 91.5%. In this embodiment,a workpiece in a complex profile, which has been hard to pick up fromthe bulked workpieces, can be picked up at high speed without performingthe adjustment for each workpiece.

According to the fifth embodiment that is configured as described above,similarly to the above-mentioned embodiments, the workpiece can bepicked up while achieving a small amount of data to be held in advanceand a short calculation time. In addition, the following advantages areobtained as well. First, the line-of-sight direction of the sensor isaligned with the entrance direction at the time when the hand approachesthe workpiece to pick up the workpiece from among the bulked workpieces.Accordingly, the optimum gripping candidate can be created at highspeed.Further, the robot is brought into a desired gripping position orposture through the restricted operation with a total of four degrees offreedom, specifically, three degrees of freedom for translation in theX-, Y-, and Z-directions and one degree of freedom for rotation aboutthe axis in the entrance direction for picking up a workpiece.Accordingly, the robot operating range, the safety range, and theinterference avoidance range can be designed easily, and further, theapparatus is activated more quickly.

Further, the optimum gripping candidate creating section uses thetwo-dimensional hand model as the hand profile data and the distanceimage as the workpiece measurement data, to thereby calculate thegripping position or posture based on the likelihood of gripping, whichis determined through convolution of the hand profile data and theworkpiece measurement data. Still further, the optimum grippingcandidate creating section extracts a flat surface or a curved surfacebased on edge detection in the distance image, and matches the flatsurface or the curved surface with the hand profile data, to therebycalculate the grippingposition or posture. Accordingly, the workpiececan be picked up at high speed, and there is no need to perform theadjustment depending on the workpiece profile, which also contributes toquicker activation of the apparatus. Yet further, the optimum grippingcandidate creating section calculates the gripping position or posturebased on the orthogonality between the opening/closing direction of thehand and the direction of the profile edge of the flat surface or thecurved surface, which is extracted based on the edge detection in thedistance image. Accordingly, the success rate of the pick-up operationfor the workpiece can be increased.

Although the above has specifically described the content of the presentinvention with reference to the preferred embodiments, it isself-evident that persons skilled in the art can adopt various kinds ofmodifications based on the basic technical concepts and teachings of thepresent invention.

Reference Signs List

1 storage meduim, 2 sensor, 3 information processing unit, 4 controlunit, 5 robot, 6 hand, 7 feed box, 30 optimum gripping candidatecreating section, 31 workpiece state determining section, 32 subsequentwork/operation estimating section, 301 grippable feature extractingsection, 302 hand matching section, 303 gripping posture candidateselecting section, 311 workpiece matching section, 312 workpieceentanglement state estimating section, 313 workpiece pick-up operationestimating section, 314 gripping posture candidate selecting section.

The invention claimed is:
 1. A workpiece pick-up apparatus, comprising:a hand for gripping a single workpiece from an object containing aplurality of bulked workpieces in a random manner; a robot for bringingthe hand into a gripping position or posture that is desired; a sensorfor performing three-dimensional measurement of the workpiece to obtainworkpiece measurement data; a storage medium for accumulating at leasthand profile data; an information processing unit for calculating thegripping position or posture based on data from the sensor and data fromthe storage medium; and a control unit for controlling the robot to gripthe workpiece based on the gripping position or posture calculated bythe information processing unit, wherein the information processing unitincludes an optimum gripping candidate creating section for directlyderiving the gripping position or posture based on the workpiecemeasurement data and the hand profile data, wherein a line-of-sightdirection of the sensor is aligned with an entrance direction at a timewhen the hand approaches the workpiece to pick up the single workpiecefrom the object containing the plurality of bulked workpieces, andwherein the optimum gripping candidate creating section uses atwo-dimensional hand model as the hand profile data and uses a distanceimage as the workpiece measurement data, to thereby calculate a grippingposition or posture at which a segment of the workpiece extracted in thedistance image is present within an opening width of the hand and thehand avoids interfering with a segment on a periphery of the workpiece.2. A workpiece pick-up apparatus according to claim 1, wherein the robotis brought into the gripping position or posture that is desired througha restricted operation with a total of four degrees of freedom, the fourdegrees of freedom comprising: three degrees of freedom for translationin an X-direction, a Y-direction, and a Z-direction; and one degree offreedom for rotation about an axis in the entrance direction for pickingup the workpiece.
 3. A workpiece pick-up apparatus according to claim 2,wherein the optimum gripping candidate creating section subtracts aregion, which is obtained through convolution of a periphery of acandidate segment of the workpiece extracted in the distance image andan interference region at a tip end of the hand, from a region, which isobtained through convolution of the candidate segment and a portion ofthe opening width of the hand, to thereby calculate the grippingposition or posture based on data obtained by smoothing.
 4. A workpiecepick-up apparatus according to claim 2, wherein the optimum grippingcandidate creating section outputs with priority a gripping position orposture which exhibits a high orthogonality between an edge direction ofthe segment and an opening/closing direction of the hand.
 5. A workpiecepick-up apparatus according to claim 1, wherein the storage mediumfurther accumulates workpiece profile data, and wherein the informationprocessing unit further comprises a workpiece state determining sectionfor evaluating, based on the workpiece profile data, the grippingposition or posture calculated by the optimum gripping candidatecreating section.
 6. A workpiece pick-up apparatus according to claim 5,wherein the workpiece state determining section comprises a workpieceentanglement state estimating section for determining interferencebetween workpiece models, which are created through matching between theworkpiece measurement data and the workpiece profile data, so as todetermine entanglement between bulked workpieces, which is not observedin the workpiece measurement data, and giving preference to a grippingposition or posture corresponding to an object with less entanglement.7. A workpiece pick-up apparatus according to claim 5, wherein theworkpiece state determining section further comprises a workpiecepick-up operation estimating section for calculating a gravity centerposition of the workpiece based on the workpiece profile data, andoutputs with priority a gripping position or posture which is closer tothe gravity center position of the workpiece.